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Abstract 

Openers, or brief activities that initiate a class, routinely take up classroom time each day yet little is 
known about how to design these activities so they contribute to student learning. This study uses 
technology-enhanced learning environments to explore new opportunities to transform Openers 
from potentially busy work to knowledge generating activities. This study compares the impact of 
teacher-designed Openers, Opener designs based on recent research emphasizing knowledge 
integration, and no Opener for an 8th grade technology-enhanced inquiry science investigation. 
Results suggest that students who participate in a researcher-designed Opener are more likely to 
revisit and refine their work, and to make significant learning gains, than students who do not 
participate in an Opener. Students make the greatest gains when they revisit key evidence in the 
technology-enhanced curriculum unit prior to revision. Engaging students in processes that promote 
knowledge integration during the Opener motivate students to revise their ideas. The results suggest 
design principles for Openers in technology-enhanced instruction. 
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Introduction 

Openers, or brief activities that initiate a class, routinely take up classroom time each day yet 
little is known about how to design these activities so they contribute to student learning. 
Teachers often give mini-lectures to remind students of what they studied in the previous 
class. Or, teachers may assign a short writing assignment. Technology-enhanced learning 
environments provide teachers with new tools to transform Openers from busy work to 
valuable learning activities. Openers can engage students in knowledge integration activities 
such as making predictions, critiquing a claim, assessing peer essays, or reflecting on 
progress—all activities that have been shown to improve student learning (Chiu and Linn, 
2011; Linn and Eylon, 2011; White and Frederiksen, 1998). This study compares the impact of 
teacher- and researcher-designed Openers to no Opener in an 8th grade technology- 
enhanced inquiry science investigation. Questions include: How effective are Openers? How 
does the design of the Opener contribute to student learning? And, what are effective post- 
Opener revision processes? 

Teacher-led Openers can play an essential role in the success of technology-enhanced 
inquiry science instruction. Several studies show that the quality of teacher implementation 
of technology-enhanced instruction predicts the impact of the technology on learning 
(Tamin, Bernard, Borokhovski, Abrami, and Schmid, 2011). In a second-order meta-analysis 
conducted on technology-enhanced instruction over the past 40 years, the authors conclude 
that the teachers' ability to monitor student understanding and help students sort out their 
ideas is essential to successful technology implementation (Williams, Linn, M, Ammon, and 
Gearhart, 2004). This meta-analysis found that as teachers asked more knowledge oriented 
questions during the course of a technology-enhanced science investigation as opposed to 
procedural questions, learning gains improved significantly. Further, to realize the potential 
of technology-enhanced instruction, teachers need to help students critically analyze 
visualizations relative to the conceptual learning goal as students often overestimate their 
understanding of dynamic computer visualizations, particularly in Chemistry (Chiu and Linn, 
2012 ). 

Technology-enhanced learning environments provide teachers with unique tools to 
structure effective Openers. The computer stores and organizes a record of each student's 
work including their multiple revisions, and provides ways to make examples of the student's 
work public to the whole class. This means teachers can purposefully select examples of the 
student's work for class discussion. Effective selection of examples could prompt students 
(and their teachers) to monitor understanding, sort out ideas about difficult concepts, and 
revisit and refine their reasoning (Izsak, 2012; Linn and Eylon, 2011). 

Using Student Work to Design Openers 

Student work is collected in technology enhanced learning environments and can be used to 
design openers. Black and Wiliam (1998) reviewed hundreds of studies and came to the 
conclusion that the use of student work for formative assessment and refinement of 
instruction is one of the most powerful ways to increase students' learning gains, particularly 
among low achieving students. Similarly, Shute (2008) reviewed work on formative feedback, 
defined as information communicated to the learner that is intended to modify his or her 
thinking or behavior to improve learning, and found that effective feedback is non- 
evaluative, supportive, timely, and specific. Gerard, Spitulnik, and Linn (2011) showed that 
when teachers use student work to refine their teaching, students' learning benefits. These 
studies distinguish between activities providing verification, where students choose whether 
an answer is correct or not, and elaboration, where teachers provide cues to guide learners 
toward a correct answer. Gerard et al found that use of student work to design instruction is 
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most effective when the resulting instruction provides advice on what to do to improve 
performance, rather than on comparisons to other students or on accuracy of responses. 

One example of a successful use of student work is for self- and peer-assessment. Research 
suggests that students can benefit from self and peer assessment activities when they 
recognize the desired learning goal, have adequate evidence to determine where the work 
stands relative to the learning goal, and have an understanding of a way to close the gap 
between the two (Black and Wiliam, 1998). Under these conditions, students can distinguish 
between their own ideas and the goal of instruction and strengthen their understanding. 
Taken together, these results suggest that effective Openers should elicit student work and 
provide hints or guidance. 

What are the challenges of formative assessment? 

Black and Wiliam's review of formative assessment studies point out that, although formative 
assessment has proven to be useful to student learning, there are concerns about how to 
enact successful formative assessment (1998). One issue is that in order to implement 
formative assessment in a useful way, teachers must have easy access to evidence of 
students' ideas and efficient routines to elicit students' ideas during class so they can help 
students to distinguish among these ideas. This calls for assessment or discussion questions 
that prompt students to make their reasoning explicit and provide multiple entry points for 
students with various levels of understanding. Students must also be actively involved in the 
feedback they are receiving in order for it to have an effect on their learning. This requires 
significant changes in a traditional secondary science classroom. Traditional science 
classroom routines often center on teacher-directed lectures and demonstrations, leaving 
little space for students to reflect on their understanding and sort out the variety of ideas 
they hold about the topic gathered from everyday experiences, peers, and school curricula. 
Ruiz-Primo and Furtak (2007), for instance found that teachers' formative assessment 
routines focused on procedural elements of inquiry learning, learning such as checking the 
students' knowledge of the correct procedure and asking them to apply a procedure to a 
new situation, rather than knowledge generation. 

How to Design an Effective Opener in Chemistry? 

This study investigates the design of knowledge integration Openers. Typical classroom 
Openers usually reflect the absorption model of instruction and use the question, response, 
evaluation (QRE) approach. In this model, the teacher asks a question with one answer in 
mind and prompts until a student gives the answer or fills in the response if none is elicited. 
This theory of learning assumes that the task of the learner is to acquire the body of 
connections that an expert analysis of the subject matter reveals (Greeno, Collins, and 
Resnick, 1996). 

In contrast, the knowledge integration perspective on learning resonates with the Black and 
Wiliam's findings (2011) and guides the design of the Openers, curriculum and assessments 
in this study. The knowledge integration perspective draws on findings from learning 
sciences research. Specifically, learners hold multiple conflicting ideas about scientific 
phenomenon as has been documented in numerous studies of student intuitions about 
science topics (diSessa, 2000). In addition, learners, often in collaboration with others, can 
deliberately sort out, link, and critique their ideas when making sense of new scientific 
phenomena and benefit from encouragement to engage in this process (Linn, Lee, Tinker, 
Husic, and Chiu, 2006; Linn and Hsi, 2000; Novak and Gowin, 1984; Slotta, Chi, and Joram, 
1995). This means that providing opportunities for students to compare alternative ideas to 
their own, develop criteria for sorting-out and distinguishing among ideas, and reflect on 
their ideas can help them form coherent hypotheses or explanations. 


81 



International Electronic Journal of Elementary Education Vol 5, Issue 1, 79-92,2012 


The knowledge integration Openers in this study were designed to help students make 
connections among their ideas about chemical reactions. In chemistry, one of the most 
difficult things for students to learn is how chemicals react. Students often have difficulties 
translating between symbolic representations, molecular representations, and observable 
phenomena (Ardac and Akaygun, 2004). Particularly, students struggle to make sense of 
chemical phenomena at the molecular level (Johnstone, 1993; Krajcik, 1991). For example, 
many students think of chemical reactions as an instantaneous process without bond 
breaking and formation, while others think all the molecules break into atoms. In addition, 
prior studies demonstrate that students often isolate molecular visualizations rather than 
linking them to existing knowledge or everyday experiences and have difficulty interpreting 
stand-alone dynamic visualizations (Tversky, Morrison, and Betrancourt, 2002; Zhang and 
Linn, 2011). This study addresses this gap in learning chemical reactions with Openers that 
ask students to reflect upon and critique peers' visual molecular representations of hydrogen 
and oxygen combustion. 

Methods 

Research Design 

This study investigates how a researcher-designed Opener, teacher-designed Opener, and a 
control condition (no Opener) contribute to students' revision of work and understanding of 
chemical reactions. Three central questions guide this research: 

1. Do Openers contribute to student understanding of chemical reactions? 

2. How does the Opener design influence students' learning outcomes? 

3. What are effective post-Opener revision processes? 

Curriculum and Assessments 

The Web-based Inquiry Science Environment (WISE) is an open-source on-line learning 
environment that includes multiple standards-aligned science inquiry curriculum units. To 
engage students in knowledge integration processes, WISE projects guide students in 
collaborative activities with visualizations of scientific phenomena that are difficult to 
observe, such as molecular views of chemical reactions (Figure 1). Students investigate 
hypotheses, design solutions to problems, critique scientific claims, and build scientific 
models, scaffolded by guidance based on knowledge integration principles. 

Students in this study worked on the WISE Hydrogen Fuel Cell Car unit. This is a one week 
unit designed to teach students about chemical reactions, alternative fuels, and energy 
(http://wise.berkeley.edu/webapp/vle/preview.html?projectld=911). The project begins by 
asking students if they would rather buy a hydrogen or gasoline powered car. It also elicits 
their ideas about energy and adds ideas about conservation of energy. Then, gasoline 
combustion in cars is explored including the relationship between carbon dioxide, a product 
of gasoline combustion, and temperature changes over the last 200 years. Students then 
create their own energy story about cars. This story includes where the energy came from to 
power the car and any chemical reactions that are involved in their story. Hydrogen 
combustion is then explored using a dynamic visualization of hydrogen combustion (Figure 
1 ). 

Then students are taught about the difference between exothermic and endothermic 
reactions and finally, students investigate a visualization of a hydrogen fuel cell to learn how 
this technology works. Students are then asked which kind of technology they would prefer 
when buying a car. 
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Assessments are embedded throughout the WISE projects to help students and teachers 
monitor student understanding and progress as students interact with visualizations (Figure 
2). The embedded assessments ask students to make predictions about the visualizations, 
sort out evidence, and link ideas together to explain their thinking. Students can also get 
hints to help them complete the tasks. 
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Figure 7. One of the visualizations in WISE's Hydrogen Fuel Cell Cars project models the 
hydrogen combustion reaction. Students can run, stop, reset, and 'spark' to start the reaction 
and explore the nature of chemical reactions. 


In the Hydrogen Fuel Cell Cars unit, an embedded assessment immediately follows the 
visualization of hydrogen combustion asking students to draw four frames of hydrogen 
combustion (Figure 2). This is meant to help students make sense of the dynamic 
visualization, recognizing features such as conservation of mass, bonds breaking, and the 
progression of the reaction. The Openers in this study focused on student work from this 
embedded assessment since in previous years this particular task was particularly 
challenging to students and yet still, understanding how to draw basic hydrogen 
combustion is critical to student understanding of chemical reactions. 


WISE >4 1 






Review Instructions | A j Frames /Snapshots) IBS 



Draw four pictures to show the combustion of six hydrogen gas molecules. The reaction 
equation is 6H 2 +30 2 ->6H 2 0 Step 1 show what the molecules look like BEFORE the 
reaction starts. Step 2: show what the molecules look like RIGHT AFTER the reaction 
starts. Step 3: show what the molecules look like LATER DURING the reaction. Step 4: 
show what the molecules look like AFTER the reaction 
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Figure2. The WISE project screen has an inquiry map on the left, a navigation bar on top, and 
an embedded assessment in the center of the screen. 
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Teacher Assessment Tools 

WISE provides teachers convenient access to a record of student work as students progress 
through a WISE unit (Figure 3). The WISE grading tools allow teachers to view embedded 
assessment data by step, and the flag tool allows teachers to select key student work 
examples for display. Students can view flagged work at any time by clicking on a tab at the 
top of their WISE screen. This allows students to actively see their peers' work and be a part 
of the feedback process to improve their understanding. Students can revise their work 
based on teachers' and peers' comments. All revisions are logged in the grading tool so the 
teacher can measure the impact of their Opener, or comments, by viewing the change in 
students' ideas from their original to revised work. 
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Figure 3. WISE assessment tool with revision number on the left, including time stamp, and 

checkbox to flag work on the right. 

Study Participants 

Two 8th grade teachers and their 236 students from 1 public school participated in this 
study. The school has medium diversity (22% receiving free lunch, 5% ELL, and 27% non¬ 
white). Both teachers had over five years experience teaching with WISE and frequently used 
Openers in their regular and WISE instruction. 

Students were randomly assigned by class period to one of the three conditions. There were 
78 students in the teacher-designed Opener, 128 students in the researcher-designed 
Opener, and 30 students in the control group condition. The uneven sample sizes were due 
to the uneven number of class periods that each teacher taught Physical Science that year. 

Opener Design 

The researcher-designed Opener engaged students in Knowledge Integration processes, as 
shown in Table 1. Activities included a small group discussion, voting, a whole group 
discussion, and then a closing summary by the teacher. The examples were selected to 
illustrate the range of conceptual errors in student representations of the chemical reaction. 
These related to (a) conservation of mass, (b) breaking of bonds, and (c) progression of the 
reaction. 
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The teacher-designed Opener alternatively focused on picking out what is good and bad 
about the student work and a lecture to try to re-teach chemical reactions from a different 
angle. Examples of student work were selected to illustrate responses that could be 
described as, "the good, the bad and the ugly". 

Table 1. Description of Openers used in Teacher 1 and Teacher 2's classroom using the 
Knowledge Integration Framework. 


Teacher-Designed 


Researcher-Designed 


Teacher 1 


Teacher 2 


Elicit Ideas: Question on the board about 
how a balanced equation obeys the law of 
conservation of mass. 

Students fill out a chart on how much mass 
(amu) exists before and after reaction. 

Add Ideas: Students use physical model of 
molecules to break bonds and put back 
together. 

Teacher shows multiple types of reactions. 
Distinguish Ideas: Teacher shows four 
examples of student work and asks 
students if the example is good or bad. 

Integrate Ideas:_ 

(~12 min) _ 

Elicit Ideas:_ 

Add Ideas: Used embedded assessment 
problem and a tree to house analogy to 
take students through the chemical 
reaction steps. 

Tree must break into parts, then 
recombine parts and build a house. 

Uses physical models of hydrogen and 
oxygen molecules to show students 
progression necessary to make water. 

Distinguish Ideas:_ 

Integrate Ideas:_ 

(~8 min) 


Students open WISE project to view 
four examples of student work in 
WISE project by clicking on the 
"Flagged Work" button. Each 
example has one unique link 
missing (Figure 4) 

Elicit Ideas: Teacher asks students 
to write down which drawing best 
represents the visualization of 
hydrogen combustion and use 
evidence to explain why. 

Students vote and teacher tallies 
votes. 

Add Ideas: Students discuss their 
choices in groups of 4 and revisit 
evidence in the visualization. 
Distinguish Ideas: Students 
reconsider their initial choice in 
light of their discussion with peers 
and revisit the evidence. Make a 
new vote. 

Integrate Ideas: Teacher tallies 
new votes and asks students to 
justify their choice. Teacher 
synthesizes criteria used to evaluate 
drawings, and instructs students to 
revise their own drawing. 

(—20 minutes) 




Link missing: conservation of mass 

Link missing: bonds breaking 
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Figure 4. Examples of student work shown during the researcher-designed Opener. 
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Data Collection and Analysis 

Data sources include: the original student work on the embedded assessment, the revised 
student work on the embedded assessment after the Opener, WISE log files illustrating 
student navigation through WISE immediately after the Opener, pre and post tests 
administered immediately before and a 1-14 days after the WISE project, classroom video of 
teacher implementation of the researcher-designed and teacher-designed Openers, and 
teacher interviews. 

Knowledge integration rubrics were used to score the embedded and pre/post assessments. 
The embedded assessment rubric is shown in Table 2. 


Table 2. Kl rubric for embedded assessment where students make step-by-step drawings of 
hydrogen combusting with oxygen 


Kl Level 

Score 

Characteristics 

Example 

Invalid 

1 

Blank or 1 don't know 



No links 

2 

No normative ideas conveyed 





but work has been done 



Simple, 1 

3 

Represents conservation of mass 



link 


OR bonds breaking OR 





progression* 



Advanced, 

4 

Conservation of mass AND 

Step 1 Draw pictures of mryecuies Before the 
reacton <€K.-JO ; -*eM.O) starts 

Step 2 Draw pictures of molecules Riffit after the 

2 links 


progression* 

0_0 

0-0 00 



OR 


(mWh) 



Conservation of mass AND bonds 





breaking 

0-® ©<£) 

0 00 



OR 

Step 3 Ora* pictures of molecules Later during the 

Step 4 Oraw pictures of molecules After the reacton 



Progression* AND bonds 

(h) ®-@ “•'.f' 




breaking 

(m> G) 

ie^ ? ' 

Complex, 

5 

Conservation of mass AND bonds 

Step 1 Drew pictures of moecuies Before the 
reacton (6H,*30,-»6H,0) Marts 

Step I Ore* pictures of molecules Right after the 
reaction starts 

3 links 


breaking AND progression* of 


0 (HI !h< 



reaction 


® ® ® 




Step 3: Ora* pictures of molecules Later during the 

Step 4 Ora* pictures of molecules After the reaction 




f"Mo' ' M 





( H 1 ®“© 

V 




@ ®-@ ® 

N/ V e' 




-k 



* Progression must be apparent on all 3 transitions 


Pre and post questions were also scored using a Kl rubric. The main ideas that students 
should understand in the pre- and post- test are that there is a reaction process that occurs 
and that H 2 and Cl 2 should start breaking bonds before forming new bonds. 

Results 

We examine the effects of Openers on student learning outcomes, and then explicate the 
contributing factors including Opener design and students' learning practices. We focus on 
embedded assessments and pretest-posttest performance. We consider the actual 
implementation of the conditions and student performance as reflected in the log files. 
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Embedded assessments 

To investigate the impact of the Opener versus no Opener on student understanding we 
compared students' original (before Opener) and revised (after Opener) responses to the 
embedded assessments. Responses were scored using the knowledge integration rubric. 
Both teacher and researcher-designed Openers (n=105 pairs) were compared to the no 
Opener condition (n=30 pairs). Pairs who did not complete the embedded assessment 
before the Opener were excluded from the analysis (n=26 pairs). Although the aim was to 
facilitate the Opener after 75% of students completed the embedded assessment, this 
number was based on an automated progress screen that only monitored whether or not 
students submitted work at least once for this assessment. Once the researchers looked at 
the student's work, it was obvious that, although work was submitted, many students did 
not finish their drawings and therefore these were not included in the data analysis. 

Responses were scored using the Kl rubric. Time stamps from the WISE log files were used to 
identify students' final work immediately before the Opener and their revised work on the 
day of the Opener. 

The analysis suggests that students who had an Opener, either teacher- or researcher- 
designed, made substantially greater learning gains than students who did not have an 
Opener. As shown in Table 4, students who had an Opener (M = .29, SD = .78) doubled the 
mean gain score of those students who did not have an Opener (M = .13, SD = .51) on the 
embedded assessment. There was no significant difference between conditions in students' 
pre-Opener scores. 

Students who had an Opener (M = .68, SD = .48) were significantly more likely to revisit and 
revise their work than students who did not have an Opener ( M = .33, SD = .48), f(131) = -3.5, 
p < 0.001. In the Opener condition, revision was twice as likely as in the no Opener condition. 


Table 3. Pre to Post Gain Scores with and without Opener. 



N Pairs Who Revised 
Their Work 

Mean Gain (SD) 

Rate of Revision (N Pairs who 
Revised/Total Pairs) 

Opener 

105 

.29(.78) 

66% 

No 

30 

.13(.51) 

33% 

Opener 





The Openers were particularly effective for students who demonstrated at least a basic 
understanding on the embedded assessment prior to the Opener (Table 7). Having an 
Opener had a significant effect on students who began with at least a partial understanding, 
or level 3 on the knowledge integration scoring rubric (M = .71, SD = .69), t(26) = -2.07, p = 
.05. Students who began with partial understanding developed their ideas into a basic 
understanding (level 4) after the Opener. In contrast, students who began with partial 
understanding and had no Opener continued to demonstrate only partial understanding 
after revising their work [M = .1 8, SD = .60). The large effect of the Opener on level 3 students 
is partially due to the examples of student work selected for critique in the researcher and 
teacher-designed Openers. The selected examples that were shown during the Opener 
illustrated characteristics of level 3 understanding, making them most accessible to this 
population of students. Since student work was chosen this way, this Opener was 
unwittingly designed to increase the learning gains of students with an already basic 
understanding of chemical reactions. 

Students with non-normative ideas, or level 2 understanding, made modest gains with or 
without an Opener, as shown in Figure 5. Level 2 students may need examples aligned with 
their own ideas to improve to partial or high level understanding. For instance, showing 
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student work with no conservation of mass or bond breaking would lead to a discussion 
where students point out that both are missing from the drawing. Alternatively, showing a 
common level 2 student work may show this population of students that their attempt is 
acknowledged and that they have direct feedback for improving their drawing. 
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Figure 5. Examples of students' work with a level 2 understanding before and after an 

Opener. 


Table 6. Pre to Post Opener Gain Scores Distributed by Pre-Opener Scores 


Pre- 

Opener 

Score 

N Pairs Who 
Revised Their 
Work 

Mean Gain 

With 

Opener(SD) 

N Pairs who Revised 

Mean Gain Without 
Opener (SD) 

2 

10 

■4(.70) 

5 

.4(.89) 

3 

17 

.71 (.69) 

11 

.18(.60)* 

4 

42 

.21(42) 

10 

0(0) 


*p=.05 


Pre-test to Post-test effects. The pre- and post- tests were administered before students began 
the week long project and after they finished the project. In spite of the gains found for the 
specific item addressed by the Opener, there was no difference on the pre/post tests 
between students who had an Opener (A/1 = 1.74, SD = 2.50) and those who did not (A/1 = 1.55, 
SD = 2.16), f(255) = -.511, p = .61. This is not surprising since the Opener treatment was only 
10-20 minutes out of the 5-7 hour long project. Perhaps several Openers over the project 
length would have been able to affect the pre test to post test results. Also, one of the 
teachers in the study did not give the post test until 2 weeks after the project was 
completed. Although the schedule was controlled, the teacher ultimately has the authority 
on when to give the post test. This delay may have masked any immediate effects of the 
Opener on the post test. Because of these time related anomalies, student performance on 
the embedded assessment is a better representation of the immediate effects of the 
Openers. 

How does the Opener design influence students' learning outcomes? 

The Opener designs, described in Table 2, differed primarily in their support for 
distinguishing and integrating ideas. In both Openers, students were presented with 
evidence regarding a chemical reaction and prompted to think about conceptual features of 
a chemical reaction. The researcher-designed Opener had additional components to support 
integration of ideas. After considering the new evidence, students were guided to reconsider 
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their initial views in light of the evidence presented, and refine their criteria for conceptual 
features of a chemical reaction. These additional components made the researcher- 
designed Opener longer (approximately 20 min) than the teacher-designed Opener 
(approximately 10 min). Since the length of both Openers was difficult to predict ahead of 
time, the students in the teacher-designed Opener did not have an additional task to 
account for the 10 minute different time on task. 

Students who participated in the researcher-designed Opener were significantly more likely 
to revise their work on the embedded assessment (M = .75, SD = .44) compared to students 
who participated in the teacher-designed Opener (M = .56, SD = .50), f(101) = -1.98, p < 0.05 
(Table 8). There was no significant difference in learning gains between the two conditions 
(Table 9). 

The researcher-designed Opener was not completely implemented as planned. The 
voting process in the researcher-designed Opener was a new activity for students and did 
not work as anticipated. In some classes, students directly copied the examples of student 
work that received the most votes even though none of the student work examples were in 
fact "correct". Although teachers reminded students that none of the student work examples 
were correct, this could have been emphasized and the teacher could have checked that 
students truly did understand that examples were not to be directly copied (have a student 
repeat it back, whole class response etc). 


Table 7. Student Kl Score (Maximum Score 5) of Embedded Assessment Before and After the 
Opener Treatment 



n 

Total 

Avg Rate of 

Pre Opener 

Post Opener 

Kl Score 


Revised 

Teams 

Revision 



Gain 

Teacher 

22 

39 

.56(50) 

3.88 

4.26 

0.38 

Designed 

Opener 

Researcher 

Designed 

Opener 

48 

64 

.75 (.44) 

3.59 

3.95 

0.36 

Table 8. Student Pair Kl Score (Out Of 5) for Pre and Post Test 


n 


Pre Test 

Post Test 

Kl Score Gain 

Teacher 

78 


2.69 

3.54 

0.85 


Designed 

Opener 

Researcher 

Designed 

Opener 

121 


2.58 

3.30 

0.72 



Log files. To analyze post-Opener learning practices we looked at log files. Two students from 
each period, one with the highest gain and one with the lowest gain, were selected for log 
file analysis in order to get an equal distribution of student learning practices from each 
condition. Students' learning practices may explain why some students made greater gains 
on the embedded assessment post Opener than others. The WISE log files show that the 
biggest contribution to learning gains was the time students spent revising their initial work, 
and revisiting relevant evidence in the WISE unit. The students who made the greatest 
improvement revisited an evidence page immediately after the Opener or spent more time 
revising their original work than other students (Table 10). An evidence page could be the 
dynamic visualization of hydrogen combustion or notes that students wrote about what a 
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good drawing of hydrogen combustion should include. This suggests the value of providing 
students with access to evidence to sort out their ideas during an Opener. 

Table 10. Comparison of Students Who Had a Gain in Their Kl Score By At Least 1 and Those 
Who Had No Gain. After The Opener, Students With a Gain Either Spent More Time on the 
Embedded Assessment or Revisited an Evidence Page After the Opener. Percentage Results Are 
Aggregated for Each Condition. 


Learning Practice 

Gain (n=6) 

No Gain (n=6) 

Revisited evidence page 

67% 

33% 

Revisited embedded assessment 
for more than 1 minute 

100% 

50% 

Both 

67% 

33% 


Discussion 


This study illustrates how Openers can improve student learning compared to not using 
Openers. Both the teacher-designed and researcher-designed Openers used in this study 
encouraged students to review their ideas. The researcher-designed Openers encouraged 
students to distinguish ideas and reflect, and resulted in a greater propensity to revise 
answers than did the teacher-designed Openers. 

These results for Openers resonate with research showing the benefit of giving students 
feedback based on their responses to assignments (Black and Wiliam, 1998; Shute, 2008). 
Openers supported students to reflect upon their initial ideas, reconsider evidence, and 
refine their views. Openers were particularly useful for students who began with a partial 
understanding of a concept. When students benefitted from Openers, they took advantage 
of the evidence presented in the Opener, revisited a dynamic visualization or another 
evidence source, and reflected on the new information. 

The findings from this study suggest the following design principles for Openers: 

1) Openers can reinforce normative conceptual ideas by drawing attention to the 
distinction between student ideas and normative views. Getting evidence from peers, 
visualization, teacher, or other classroom resource can help students understand complex 
ideas. This evidence helps students close the gap between what they know now and the 
normative view of the phenomena studied. 

2) Openers help students when they occur soon after a topic is introduced and direct 
attention to specific ways to improve their ideas. 

3) Openers succeed when they are non-evaluative and support students to explore 
evidence or views of their peers. For example, teachers can support students by giving them 
the opportunity to distinguish among the ideas held by the group of students in the class. 
Giving students a chance to appreciate conflicting views held by classmates and use 
evidence to sort them out helps students integrate their ideas. 

4) Openers should be short. The Openers in this study were much longer and had more 
teacher involvement for most Openers. It may work better to have students review examples 
of student work as homework, then discuss with their partner at the very start of class. This 
would free the teacher up to take attendance and give more time for the peer and class 
discussion. 


• • • 
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